Upgrade Couchbase support for Spring Boot 2.4 by mraible · Pull Request #13770 · jhipster/generator-jhipster

mraible · 2021-01-30T00:57:32Z

Please make sure the below checklist is followed for Pull Requests.

All continuous integration tests are green
Tests are added where necessary
jhipster-online was updated if necessary
Documentation is added/updated where necessary
Coding Rules & Commit Guidelines as per our CONTRIBUTING.md document are followed

When you are still working on the PR, consider converting it to Draft (bellow reviewers) and adding skip-ci label, you can still see CI build result at your branch.

These serious bugs are already fixed in 4.1.0-M1 so we used that version until 4.0.2 is released

…Repository that provides the same methodes

Solves CI error: 'dependencies.dependency.version' for com.couchbase.client:encryption:jar is missing. @ line 151, column 21

https://docs.spring.io/spring-data/couchbase/docs/current/reference/html/#couchbase.migrating.entities

Co-authored-by: Marcelo Shima <marceloshima@gmail.com>

mshima · 2021-01-30T01:55:41Z

ejs problem is fixed.

https://docs.spring.io/spring-data/couchbase/docs/4.1.3/reference/html/#repositories.core-concepts

… into fix-couchbase

mraible · 2021-01-30T02:48:35Z

It seems like there's a lot to do to upgrade to Spring Data Couchbase 4.x. I give up and will add a bug bounty to the open issue for this PR.

mraible · 2021-01-30T03:20:05Z

You can use JHipster Daily Builds + the Couchbase job to trigger a build for your branch. If you don't have access, please let me know and I can trigger it for you.

mshima · 2021-01-30T12:39:17Z

Couchbase is completely broken, but we should merge to fix ejs template.

mraible · 2021-01-30T17:59:35Z

I merged in @murdos' changes. Testcontainers doesn't seem to like the Docker container.

[ERROR] tech.jhipster.sample.web.rest.errors.ExceptionTranslatorIT  Time elapsed: 0.006 s  <<< ERROR!
java.lang.IllegalStateException:
Failed to verify that image 'couchbase:6.6.1' is a compatible substitute for 'couchbase/server'. This generally means that you are trying to use an image that Testcontainers has not been designed to use. If this is deliberate, and if you are confident that the image is compatible, you should declare compatibility in code using the `asCompatibleSubstituteFor` method. For example:
   DockerImageName myImage = DockerImageName.parse("couchbase:6.6.1").asCompatibleSubstituteFor("couchbase/server");
and then use `myImage` instead.

I tried changing it from couchbase:6.6.1 to couchbase/server in CouchbaseTestContainerExtension, but tests still fail.

[ERROR] Tests run: 341, Failures: 89, Errors: 25, Skipped: 0

mraible · 2021-01-30T18:02:20Z

This seems to fix the issue, but there's still a lot of test failures.

            DockerImageName image = DockerImageName.parse("couchbase:6.6.1").asCompatibleSubstituteFor("couchbase/server");
            couchbase =
                new CouchbaseContainer(image)
                    .withBucket(new BucketDefinition(getBucketName()).withQuota(100))
                    .withCredentials("user", "secret");

mraible · 2021-01-30T23:11:12Z

Down to 9 test failures for the ngx-couchbase project!

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   UserServiceIT.assertThatNotActivatedUsersWithNotNullActivationKeyCreatedBefore3DaysAreDeleted:144
Expecting actual not to be empty
[ERROR]   UserServiceIT.assertThatUserCanResetPassword:126
Expecting Optional to contain a value but it was empty.
[ERROR]   UserServiceIT.assertThatUserMustExistToResetPassword:72
Expecting Optional to contain a value but it was empty.
[ERROR]   AccountResourceIT.testActivateAccount:380 Status expected:<200> but was:<500>
[ERROR]   AccountResourceIT.testFinishPasswordReset:688 Status expected:<200> but was:<500>
[ERROR]   AccountResourceIT.testRegisterDuplicateLogin:259
Expecting Optional to contain a value but it was empty.
[ERROR]   AccountResourceIT.testSaveExistingEmail:484 Status expected:<400> but was:<200>
[ERROR]   PublicUserResourceIT.getAllPublicUsers:55 Status expected:<200> but was:<500>
[ERROR]   UserResourceIT.updateUserExistingEmail:399 Status expected:<400> but was:<200>
[ERROR] Errors:
[ERROR]   DomainUserDetailsServiceIT.assertThatUserCanBeFoundByEmailIgnoreCase:96 » UsernameNotFound
[INFO]
[ERROR] Tests run: 341, Failures: 9, Errors: 1, Skipped: 0

Time for a bike ride. :)

Based on this advice: spring-projects/spring-data-couchbase#897 (comment)

mraible · 2021-01-31T01:50:49Z

Now we're at 6 failures and 1 error after the last commit that adds @ScanConsistency to UserRepository.

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   UserServiceIT.assertThatNotActivatedUsersWithNotNullActivationKeyCreatedBefore3DaysAreDeleted:144
Expecting actual not to be empty
[ERROR]   UserServiceIT.assertThatUserCanResetPassword:126
Expecting Optional to contain a value but it was empty.
[ERROR]   AccountResourceIT.testActivateAccount:380 Status expected:<200> but was:<500>
[ERROR]   AccountResourceIT.testFinishPasswordReset:688 Status expected:<200> but was:<500>
[ERROR]   FieldTestPaginationEntityResourceIT.fullUpdateFieldTestPaginationEntityWithPatch:1244
Expecting:
 <"AAAAAAAAAA">
to be equal to:
 <"BBBBBBBBBB">
but was not.
[ERROR]   PublicUserResourceIT.getAllPublicUsers:55 Status expected:<200> but was:<500>
[ERROR] Errors:
[ERROR]   DomainUserDetailsServiceIT.assertThatUserCanBeFoundByEmailIgnoreCase:96 » UsernameNotFound
[INFO]
[ERROR] Tests run: 341, Failures: 6, Errors: 1, Skipped: 0

mraible · 2021-01-31T17:14:18Z

I think if we could configure default consistency for tests, we could fix a lot of these failures. This SO question makes it look like it was possible in previous versions.

e2e tests pass too!

mraible · 2021-02-04T21:29:32Z

There's still a lot of tests failing for the reactive implementation of Couchbase:

[ERROR] Failures:
[ERROR]   UserServiceIT.assertThatNotActivatedUsersWithNotNullActivationKeyCreatedBefore3DaysAreDeleted:146
Expecting actual not to be empty
[ERROR]   UserServiceIT.assertThatUserCanResetPassword:125
Expecting Optional to contain a value but it was empty.
[ERROR]   AccountResourceIT.testActivateAccount:458 Status expected:<200 OK> but was:<500 INTERNAL_SERVER_ERROR>
[ERROR]   AccountResourceIT.testFinishPasswordReset:809 Status expected:<200 OK> but was:<500 INTERNAL_SERVER_ERROR>
[ERROR]   EntityWithPaginationAndDTOResourceIT.getAllEntityWithPaginationAndDTOS:145 JSON path "$.[*].id"
Expected: a collection containing "entitywithpaginationanddto__568aa2c1-d996-4edc-8b58-d9661e486ceb"
     but: was empty
[ERROR]   EntityWithServiceClassPaginationAndDTOResourceIT.getAllEntityWithServiceClassPaginationAndDTOS:151 JSON path "$.[*].id"
Expected: a collection containing "entitywithserviceclasspaginationanddto__453a44d5-e765-4996-aba2-9948d354be34"
     but: was empty
[ERROR]   EntityWithServiceImplPaginationAndDTOResourceIT.getAllEntityWithServiceImplPaginationAndDTOS:151 JSON path "$.[*].id"
Expected: a collection containing "entitywithserviceimplpaginationanddto__fddd7d49-74b9-46a3-8fed-79e58cac9465"
     but: was empty
[ERROR]   FieldTestInfiniteScrollEntityResourceIT.getAllFieldTestInfiniteScrollEntities:803 JSON path "$.[*].id"
Expected: a collection containing "fieldtestinfinitescrollentity__a3ffee23-3585-4f00-b6da-9cd8cb163199"
     but: was empty
[ERROR]   FieldTestPaginationEntityResourceIT.getAllFieldTestPaginationEntities:758 JSON path "$.[*].id"
Expected: a collection containing "fieldtestpaginationentity__2699c14f-90db-48e7-9508-6b2cd535dad9"
     but: was empty
[ERROR]   PublicUserResourceIT.getAllPublicUsers:57 Status expected:<200 OK> but was:<500 INTERNAL_SERVER_ERROR>
[ERROR] Errors:
[ERROR]   UserResourceIT.getAllUsers:255 NullPointer

Test run using daily builds: https://github.com/hipster-labs/jhipster-daily-builds/actions/runs/538490717

mraible · 2021-02-05T16:09:46Z

Do we need to set up testcontainers to support utf-8 for PostgreSQL? Error from webflux-psql job:

2021-02-05 16:00:18.036  WARN 3047 --- [tor-tcp-epoll-2] i.r.p.client.ReactorNettyClient          : 
Error: SEVERITY_LOCALIZED=ERROR, SEVERITY_NON_LOCALIZED=ERROR, CODE=22021, MESSAGE=invalid byte 
sequence for encoding "UTF8": 0x00, FILE=mbutils.c, LINE=1597, ROUTINE=report_invalid_encoding
Error:  Tests run: 13, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.163 s <<< FAILURE! - 
in tech.jhipster.sample.web.rest.MapsIdUserProfileWithDTOResourceIT
Error:  tech.jhipster.sample.web.rest.MapsIdUserProfileWithDTOResourceIT.updateMapsIdUserProfileWithDTOMapsIdAssociationWithNewId  Time elapsed: 0.114 s  <<< ERROR!
org.springframework.r2dbc.BadSqlGrammarException: executeMany; bad SQL grammar 
[INSERT INTO jhi_user (login, password_hash, first_name, last_name, email, activated, lang_key, image_url, created_by, created_date, last_modified_date) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)]; 
nested exception is io.r2dbc.postgresql.ExceptionFactory$PostgresqlBadGrammarException: 
[22021] invalid byte sequence for encoding "UTF8": 0x00
Caused by: io.r2dbc.postgresql.ExceptionFactory$PostgresqlBadGrammarException: 
invalid byte sequence for encoding "UTF8": 0x00

mraible · 2021-03-05T19:35:13Z

Closing and reopening to run CI again.

mraible · 2021-03-05T19:40:17Z

I had to re-create this PR because I deleted my fork. See #14184.

murdos and others added 5 commits August 24, 2020 17:21

Couchbase: Update to Spring Data Couchbase 4.0 and Couchbase SDK 3

c6cac46

Workaround for DATACOUCH-546 and DATACOUCH-576 bugs

02b417a

These serious bugs are already fixed in 4.1.0-M1 so we used that version until 4.0.2 is released

Remove generated N1qlCouchbaseRepository in favor of Spring Couchbase…

b106334

…Repository that provides the same methodes

Fix deprecated media type

7c28f62

Attempt to fix Couchbase CI failure

683806c

mraible requested a review from mshima January 30, 2021 00:57

mraible added 3 commits January 29, 2021 18:06

Revert to getId() for Couchbase

658ed1d

Add version for com.couchbase.client.encryption.

5456593

Solves CI error: 'dependencies.dependency.version' for com.couchbase.client:encryption:jar is missing. @ line 151, column 21

Add version to Couchbase java-client

6f0d31b

mraible mentioned this pull request Jan 30, 2021

Add version configuration for Couchbase encryption jhipster/jhipster-bom#149

Merged

mshima requested changes Jan 30, 2021

View reviewed changes

Comment thread generators/entity-server/templates/src/main/java/package/domain/Entity.java.ejs Outdated

Comment thread generators/entity-server/templates/src/main/java/package/domain/Entity.java.ejs Outdated

mraible and others added 3 commits January 29, 2021 18:38

Update imports for Spring Data Couchbase

878bf48

https://docs.spring.io/spring-data/couchbase/docs/current/reference/html/#couchbase.migrating.entities

Update primaryKey logic based on code review

fd6516f

Co-authored-by: Marcelo Shima <marceloshima@gmail.com>

Update primaryKey logic based on code review

52e9f83

Co-authored-by: Marcelo Shima <marceloshima@gmail.com>

mshima requested changes Jan 30, 2021

View reviewed changes

Comment thread generators/entity-server/templates/src/main/java/package/domain/Entity.java.ejs Outdated

Comment thread generators/entity-server/templates/src/main/java/package/domain/Entity.java.ejs Outdated

Fix otherEntity primary key

fbca067

mraible added 2 commits January 29, 2021 18:57

Update class for Spring Data Couchbase

e9e23c5

https://docs.spring.io/spring-data/couchbase/docs/4.1.3/reference/html/#repositories.core-concepts

Merge branch 'fix-couchbase' of github.com:mraible/generator-jhipster…

1a69554

… into fix-couchbase

Fix imports

878bcff

mraible mentioned this pull request Jan 30, 2021

Couchbase tests are failing #13544

Closed

pascalgrimaud closed this in jhipster/jhipster-bom#149 Jan 30, 2021

mshima reopened this Jan 30, 2021

mshima approved these changes Jan 30, 2021

View reviewed changes

mraible added 3 commits January 30, 2021 08:22

Remove encryption version since it's managed by jhipster-bom.

1ecac87

Remove workarounds for Spring Data Couchbase

66b9f1e

Merge branch 'main' of github.com:mraible/generator-jhipster into main

cc06968

mraible added 3 commits January 30, 2021 09:56

Merge branch 'spring-boot-2.3-couchbase' into fix-couchbase

8a516e5

Update based on @murdos' spring-boot-2.3-couchbase branch

1957351

Polishing after testing

c416550

mraible added 3 commits January 30, 2021 11:03

Fix Docker image

cc978c9

Fix typo in test

625389b

Fix tests by adding @ScanConsistency to findAll() methods

82e8b59

Add @ScanConsistency to all findAll*() methods

f828926

Based on this advice: spring-projects/spring-data-couchbase#897 (comment)

mraible added 4 commits January 31, 2021 13:06

Fix Couchbase + reactive so it compiles and runs

329edf8

e2e tests pass too!

Integrate @mmonti's changes from mraible/ngx-couchbase#1

0cbee31

Import org.springframework.data.domain.Page for everyone

a504560

Fix javadoc

c6031f1

DanielFran requested changes Feb 4, 2021

View reviewed changes

mraible changed the title ~~Attempt to fix Couchbase CI failure~~ Upgrade Couchbase support for Spring Boot 2.4 Feb 4, 2021

mraible added 2 commits February 4, 2021 12:42

Updates after code review

99e4507

Try to fix reactive couchbase

f857f0b

mraible added 2 commits February 4, 2021 14:44

Fix User types

a930ad8

Fix Couchbase file location test

46d6ce1

mraible closed this Mar 5, 2021

mraible mentioned this pull request Mar 5, 2021

Upgrade Couchbase support for Spring Boot 2.4 #14184

Closed

5 tasks

pascalgrimaud added this to the v7.0.0-beta.2 milestone Mar 8, 2021

gkysaad mentioned this pull request Jul 14, 2021

Couchbase SDK 3, Spring Data Couchbase 4 Integration #15645

Merged

6 tasks

Uh oh!

Conversation

mraible commented Jan 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mshima commented Jan 30, 2021

Uh oh!

mraible commented Jan 30, 2021

Uh oh!

mraible commented Jan 30, 2021

Uh oh!

mshima commented Jan 30, 2021

Uh oh!

mraible commented Jan 30, 2021

Uh oh!

mraible commented Jan 30, 2021

Uh oh!

mraible commented Jan 30, 2021

Uh oh!

mraible commented Jan 31, 2021

Uh oh!

mraible commented Jan 31, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mraible commented Feb 4, 2021

Uh oh!

mraible commented Feb 5, 2021

Uh oh!

mraible commented Mar 5, 2021

Uh oh!

mraible commented Mar 5, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mraible commented Jan 30, 2021 •

edited

Loading